Evaluating the effects of simulation training on stroke thrombolysis: a systematic review and meta-analysis

Background Ischaemic strokes are medical emergencies, and reperfusion treatment, most commonly intravenous thrombolysis, is time-critical. Thrombolysis administration relies on well-organised pathways of care with highly skilled and efficient clinicians. Simulation training is a widespread teaching modality, but results from studies on the impact of this intervention have yet to be synthesised. This systematic review and meta-analysis aimed to synthesise the evidence and provide a recommendation regarding the effects of simulation training for healthcare professionals on door-to-needle time in the emergency thrombolysis of patients with ischaemic stroke. Methods Seven electronic databases were systematically searched (last updated 12th July 2023) for eligible full-text articles and conference abstracts. Results were screened for relevance by two independent reviewers. The primary outcome was door-to-needle time for recombinant tissue plasminogen activator administration in emergency patients with ischaemic stroke. The secondary outcomes were learner-centred, improvements in knowledge and communication, self-perceived usefulness of training, and feeling ‘safe’ in thrombolysis-related decision-making. Data were extracted, risk of study bias assessed, and analysis was performed using RevMan™ software (Web version 5.6.0, The Cochrane Collaboration). The quality of the evidence was assessed using the Medical Education Research Study Quality Instrument. Results Eleven studies were included in the meta-analysis and nineteen in the qualitative synthesis (n = 20,189 total patients). There were statistically significant effects of simulation training in reducing door-to-needle time; mean difference of 15 min [95% confidence intervals (CI) 8 to 21 min]; in improving healthcare professionals’ acute stroke care knowledge; risk ratio (RR) 0.42 (95% CI 0.30 to 0.60); and in feeling ‘safe’ in thrombolysis-related decision-making; RR 0.46 (95% CI 0.36 to 0.59). Furthermore, simulation training improved healthcare professionals' communication and was self-perceived as useful training. Conclusion This meta-analysis showed that simulation training improves door-to-needle times for the delivery of thrombolysis in ischaemic stroke. However, results should be interpreted with caution due to the heterogeneity of the included studies. Supplementary Information The online version contains supplementary material available at 10.1186/s41077-024-00283-6.


Background
Stroke is the second-leading cause of mortality worldwide [1].The vast majority of strokes have an ischaemic pathogenesis [2][3][4] though underlying mechanisms may be variable and complex [3].From the onset of clinical symptoms, the ischaemic core is surrounded by neurons that may remain viable for several hours prior to the development of irreversible ischaemic injury [5,6].This affords a treatment window where prompt restoration of blood supply may permit the survival of the threatened neurons, known as the penumbra [6][7][8][9].The determinants of whether cerebral ischaemia leads to infarction are anatomical (relating to the presence and extent of protective collateral circulation) and time-critical, with respect to having access to reperfusion treatment [10,11].Stroke patients identified early have the greatest potential to benefit from reperfusion either via mechanical thrombectomy (currently restricted mainly to specialised centres) [12] but more commonly with intravenous thrombolysis using recombinant tissue plasminogen activator (rtPA), through therapeutic benefit diminishes with time [6][7][8][9].
Consequently, emphasis on awareness of stroke symptoms and the time-critical nature of assessment has increased through organisations like Brain Attack Coalition [13] and public health campaigns.The adoption of tools such as Facial drooping, Arm weakness, Speech difficulties, and Time (FAST), now used widely by paramedics, improves recognition and enables pre-alert of the receiving hospital to patient arrival [14].Reduced time from the hospital door to rtPA administration (door-to-needle time) [15] alone decreases mortality and haemorrhagic transformation associated with ischaemic stroke [6,16], with a target of under 60 min set internationally [17][18][19].Therefore, clinical pathways for emergency stroke patients have to be responsive and efficient throughout, from first notification by ambulance services to the Emergency Department (ED), patient reception, computed tomography (CT) imaging, through to obtaining specialist radiology and clinical assessment to determine the best course of action, with the aim of swiftly initiating intravenous thrombolysis if appropriate [20].Barriers to administering rtPA to those patients who may benefit include clinician uncertainty regarding the administration of treatment with the potential to cause harm and lack of practice in delivery [21,22].Current stroke guidelines urge the establishment of educational initiatives to improve outcomes in patients presenting as emergencies with ischaemic stroke [23].
Simulation training has been widely used as an educational modality in several specialties with "timedependent" processes such as trauma care and life support [24][25][26].However, adoption has been slower within the neurological sciences [27], and evidence suggests that human factors are the most significant ratelimiting component in the delivery of emergency care to stroke patients [28].In this clinical context, simulation training may provide an opportunity for teams to increase knowledge and develop the processes, skills, and teamwork required to optimise the safe delivery of intravenous thrombolysis in educationally beneficial representations of real-world environments [12,29].
The effectiveness of simulation training on the investigated outcomes can be assessed using Kirkpatrick's Four-Level Training Evaluation model, which identifies the effects of particular training on the organisation level and patients as a whole [30], and is considered the reference standard for evaluation of training in healthcare contexts [31].
Although there are numerous primary studies on the effects of simulation training on door-to-needle time, to the authors' knowledge, no meta-analysis on this topic exists.This systematic review and meta-analysis aimed to address the gap in the literature by assessing the effects of simulation training for healthcare professionals on door-to-needle time delivery of emergency thrombolysis in ischaemic stroke.

Study design
This systematic review and meta-analysis were performed per the guidelines of the Cochrane Handbook of Systematic Reviews of Interventions [32] and the Preferred Reporting Items for Systematic Review and Meta-Analysis (PRISMA) statement [33].Appendix 1 details the PRISMA checklist.The objective was to synthesise the available evidence regarding the effects of simulation training for healthcare professionals on door-to-needle time delivery of emergency thrombolysis in ischaemic stroke patients.

Study eligibility
Any study investigating healthcare professional simulation training with respect to intravenous thrombolysis administration in stroke patients versus no intervention was eligible for inclusion, with the primary outcome being door-to-needle time and learner-centred secondary outcomes.Simulation training or activity was defined as the complete set of events and actions that occur from initiation to termination of a particular simulation event [34].No intervention was defined as any period without simulation training.
Table 1 illustrates the eligibility criteria.

Study identification
The literature search was first conducted on 17th May 2023 and last updated on 12th July 2023 using EMBASE, PubMed, PsycINFO, ERIC, CINAHL, Scopus, and Google Scholar.The entry date was 1990 when the results of the first recombinant tissue plasminogen activator (rtPA) trial were published, which was followed by the United States Food and Drug Administration's approval for rtPA as a treatment for acute ischaemic stroke in 1996 [35].
The search was performed by two independent researchers (SA and AA).The search strategy included MeSH and text search terms, agreed upon by the research team, combined with Boolean operators "AND" and "OR".Details of the search strategies are listed in Appendix 2.
Electronic search strategies were limited to adult humans (over 18 years old), and no restrictions on language or publication types were applied.Citation lists of included publications were manually scrutinised for additional relevant studies, and a manual search of international conference abstract databases was performed, including the Association for Simulated Practice in Healthcare [36], Society for Simulation in Europe [37], International Meeting on Simulation in Healthcare [38], Australasian Simulation Congress [39], and the International Clinical Skills Conference [40].

Study selection
All titles and abstracts retrieved were independently screened for relevance.Using Microsoft Excel ® , duplicates were manually removed, and non-relevant articles were excluded.The full texts of all identified studies were retrieved and assessed for eligibility independently by two researchers (SA and AA).Studies meeting the eligibility criteria were included following the Cochrane Handbook for Systematic Reviews of Interventions.Discrepancies between SA and AA were discussed and agreed upon with a senior reviewer (AM), ensuring no potentially relevant papers were discarded. 32

Data extraction
Data were extracted based on the guidelines for health care simulation research [41] and inputted into a Microsoft Excel ® spreadsheet (Appendix 3).Non-English articles were translated completely.All corresponding authors of included studies with any missing data were contacted.

Data synthesis
Quantitative analysis was performed using RevMan ™ (Web version 5.6.0,The Cochrane Collaboration) [42].A meta-analysis was performed for the primary outcome, with results expressed as mean difference with 95% confidence intervals (CI) and secondary outcome results expressed as risk ratio (RR) with 95% CI.Where possible, median and interquartile range (IQR) values were converted to mean (x) , and standard deviation (SD) using the formulae [43] (Q 1 = 25th and Q 3 = 75th percentiles).A p-value of < 0.05 was considered statistically significant.Heterogeneity between studies was assessed by the I 2 score, using the random-effects meta-analysis model to account for data heterogeneity [32].Two sensitivity analyses were performed for the primary outcome.The first included studies eligible for meta-analysis with low-to-moderate risk of bias and the second, studies of moderate-to-high methodological quality.Subgroup analysis was not performed for the primary outcome due to insufficient details on patient characteristics being available.

Assessment of risk of bias and study quality
The Cochrane 'Risk of Bias in Non-Randomised Studies of Interventions' (ROBINS-I) tool was used to assess the risk of bias [44], with graphs generated by Robvis [45].Appendix 4 includes details of the ROBINS-I assessment.The quality of the included studies was assessed using the Medical Education Research Study Quality Instrument (MERSQI); low-quality studies scored MERSQI ≤ 12 [46].
Three studies utilised Crew Resource Management aspects in reducing door-to-needle times [47,52,63].Crew Resource Management is a training concept focusing on non-technical and behavioural skills such as situational awareness, decision-making, leadership, and teamwork [66,67].
Table 2 presents the characteristics of the included studies.Appendix 5 presents a list of excluded studies (n = 32 studies).

Outcomes
The outcomes are divided into primary and secondary outcomes.

Secondary outcomes
Improvement in acute stroke knowledge Four studies [47,51,52,55] assessed improvement in knowledge through surveys, reporting improvements ranging from 46.6% pre-simulation training to 84.6% post-simulation training.Meta-analysis showed a statistically significant effect in favour of post-simulation training in improving healthcare professionals' acute stroke knowledge, with a pooled RR of 0.42 (95% CI 0.30 to 0.60) (Fig. 5).Feeling 'safe' in thrombolysis-related decision-making Two studies [47,52] assessed healthcare professionals' feelings of 'safety' in thrombolysis-related decisionmaking, reporting improvements ranging from 26.7 to 74.3%.Meta-analysis showed a statistically significant effect favouring post-simulation training with respect to improved feelings of safety in thrombolysis-related decision-making, with a pooled RR of 0.46 (95% CI 0.36 to 0.59) (Fig. 6).

Publication bias
A Funnel plot of the eleven studies included in the metaanalysis with respect to door-to-needle time was generated.Visual inspection reveals asymmetry (Fig. 7).

Risk of Bias (ROBINS-I) in included studies
The risk of bias was assessed using a ROBINS-I risk-ofbias graph (Fig. 8A, B).Most studies had good participant selection of healthcare professionals and outcome measurements, with low deviations from intended interventions.Bias from confounding, missing data, and selective  reporting of were difficult to assess due to insufficient information.One study had an overall low risk of bias [57], eight had a moderate risk of bias [47,49,51,52,55,56,58,63], two had a serious risk of bias [60,65], and eight had insufficient information [48,50,53,54,59,61,62,64].

Heterogeneity
No studies were excluded based on methodological heterogeneity.There was a high estimate of statistical heterogeneity (I 2 = 98%) in those studies included in our primary analysis with respect to door-to-needle time for intravenous thrombolysis in ischaemic stroke patients (Fig. 2).

Discussion
In this systematic review and meta-analysis, we found that simulation had a beneficial effect, reducing doorto-needle time for the emergency delivery of intravenous thrombolysis in patients with ischaemic stroke by about 15 min.For each minute in a middle cerebral artery stroke without treatment, around 1.9 million neurons are lost [68].Each 15-min reduction in delay to treatment may achieve a 4% increase in good clinical outcomes [69].As such, numerous guidelines are advocating for a downward revision in door-to-needle target times, aiming for under 30 min [70].Given the wellestablished relationship between early recombinant tissue plasminogen activator (rtPA) administration and improved patient outcomes [70], an improvement of this magnitude has important clinical implications.Despite notable methodological and statistical heterogeneity among the included studies, sensitivity analysis corroborated the collective impact favouring post-simulation training.Methodological variances arose from several sources.The studies were conducted across ten different countries, each with particular healthcare system nuances.Though international stroke care standards are recognised, these may be harder to fulfil in some settings than others, and the areas for gain with regard to improving door-to-needle times may vary greatly.For example, delays may be apparent to a greater or lesser extent at different points in the patient journey, dependent on resources such as a dedicated stroke physician or access to computed tomography (CT) scanning.Such variations would be challenging to control for, even in multicentre randomised controlled trials.Moreover, study participants varied significantly in terms of numbers and healthcare professions, including physicians of various levels of experience, from stroke care specialists to radiologists, paramedics, and nurses.There was also diversity in the approaches to conducting simulations, both in terms of the environments the simulations were conducted in, ranging from in situ simulations to classroombased scenarios, and how patients were represented While implementing simulation training, five studies concurrently revised their stroke protocol, raising the possibility of validation bias, and three used multifaceted interventions, increasing confounding and difficulty attributing complete improvements in door-to-needle time to simulation training alone.However, simulation training cannot be considered a stand-alone activity in itself but rather one part of a multi-faceted approach to quality improvement, systems redesign and testing, and team/organisational culture [71,72].Translational simulation for transformative rather than pedagogical purposes is being recognised as a growing field as a means to promote change within clinical systems [73][74][75].Considering that simulation training is an entanglement of activities in process and quality improvement, such as testing new protocols, a different sensitivity analysis, and separating studies with concurrent interventions, was not conducted.
Despite the explained methodological diversity across the studies, there was adequate consistency in reported outcomes related to door-to-needle time and learner experiences to synthesise the results.Our findings align with existing literature on the application of simulation training to enhance aspects of care in other emergency patient scenarios, such as cardiac arrest [76], cardiac catheterisation [77], extracorporeal membrane oxygenation [78], and maternal cardiac arrest [79].
With respect to our learner-centred secondary outcomes, we found that stroke subject knowledge, clinical perception of safety in thrombolysis decision-making, self-perceived usefulness, and communication were all found to increase after simulation training, mirroring the existing literature from other clinical environments [80][81][82].Drawing a direct causative relationship is not possible for these qualitative outcomes, but it seems likely that the suitability of simulation training for understanding and improving macro-ergonomics and human factors is important [83].
The successful and timely administration of intravenous thrombolysis involves highly complex systems and multidisciplinary teams.Expert consensus acknowledges simulation training as particularly beneficial for testing, practising, executing, and evaluating peri-operative microsystems [84].These microsystems are also highrisk, complex hospital systems involving large multidisciplinary teams caring for patients having surgical procedures, and our findings support the possibility that this also applies to acute stroke management.Additionally, simulation training is well-recognised for its benefits in multidisciplinary team education [85,86], which may contribute to our findings of an overall reduction in door-to-needle time.Simulation training in neurocritical care has been slower to gain acceptance compared to other medical disciplines [87].This may be partly due to the challenge of replicating time-sensitive neurological emergencies like stroke and status epilepticus using simulation manikins or actors [86,87].
Kirkpatrick's model for evaluation of training is a fourstep model which categorises learning outcomes into four levels: (1) reaction, (2) learning, (3) behaviour, and (4) results [88].In clinical environments, Level 4 outcomes can be represented by patient-centred outcomes.This review implies that both Level 2 (learner-reported outcomes) and Level 4 outcomes (door-to-needle time) align with Kirkpatrick's scale.A recent systematic review that covered a wide range of medical education simulations only identified 13 studies for inclusion and reported a paucity of studies employing Kirkpatrick's Level 4 outcomes to evaluate simulation training [89].
While the overall quality of the studies included in this review is, at best, moderate, it is worth noting that this study is not the first to question the quality of studies in simulation training.Even when reviewing randomised controlled trials of simulations [90], authors found a high risk of bias (86%), a lack of reported findings (4%), an absence of registered protocols, as well as various issues with blinding and concealment.While simulation training appears to be a valuable technique in various aspects of medical education, there is sufficient equipoise for further high-quality, standardised studies evaluating Kirkpatrick's Level 4 outcomes.

Strengths
This meta-analysis is the first to assess the effects of simulation training on door-to-needle time regarding emergency thrombolysis delivery to patients with ischaemic stroke and has important strengths.First, it was conducted according to PRISMA guidelines [32] and encompassed a comprehensive search strategy with no language restrictions.Second, the authors of all included studies with missing data were contacted regarding the existence of any other data to ensure methodological robustness.Third, methodological data extraction utilised a customcreated extraction sheet.Fourth, the ROBINS-I tool was applied to accurately assess the risk of bias, facilitating the estimation of the true effects of simulation training [44].Lastly, two sensitivity analyses were performed to ensure the quality and robustness of the results [91].

Limitations
The limitations of this meta-analysis primarily pertain to the weaknesses of the source articles.First, the limited number of studies and variance in methods (only one was a randomised control study), sample sizes, study periods, and number of strokes may influence the validity of the findings.We found a high degree of heterogeneity among the included studies, and though we used a random-effects model to account for this, we recognise this as a weakness, affecting the strength of our conclusions.Second, the concurrent introduction of simulation training and revision of stroke protocols in some studies raises the possibility of validation bias.While this is important for meta-analysis, simulation training is not a stand-alone activity and, therefore, must be interpreted within the context of quality improvement.Third, the critical appraisal using the ROBINS-I tool was performed by one independent reviewer, which may have increased the risk of bias.Fourth, the asymmetry of the funnel plot indicates heterogeneity and publication bias among the included studies.Fifth, translational simulation was not included as a search term, and therefore, the search strategy may have failed to capture evidence on this aspect.Sixth, the reported medians and interquartile ranges (IQR) were converted to means and standard deviations (SD), respectively, which may have introduced bias and imprecise estimates.Seventh, a sensitivity analysis separating studies with concurrent interventions was not performed due to simulation training being an entanglement of activities in process and quality improvements, which may have increased the risk of bias when assessing the pure effect of simulation.Lastly, we acknowledge that this review was not registered with PROSPERO.

Future directions
There is a need for robust and standardised multiinstitutional studies with randomised controlled trial designs using Kirkpatrick's Level 4 outcomes and larger sample sizes, inclusive of all healthcare professionals involved in the delivery of emergency thrombolysis in ischaemic stroke.Future studies should standardise the reporting of simulation-based interventions using standardised reporting tools [41], clearly demarcating its different types using rigorous and reproducible outcome measures, namely the Kirkpatrick Model [92].Future work should consider that simulation for healthcare improvement is one part of any contemporary quality improvement strategy, and as such, the results of simulation-based studies need to be interpreted and considered alongside simultaneous contextual changes such as protocol refinements or process changes within complex systems.

Conclusion
This meta-analysis showed a significant beneficial effect of simulation training in reducing door-to-needle time delivery of emergency thrombolysis in ischaemic stroke.Additionally, simulation training was associated with improved knowledge, communication, and a feeling of 'safety' in thrombolysis-related decision-making.The results should be interpreted with caution due to the heterogeneity of the included studies.Further high-quality research is warranted to strengthen the evidence base and establish confidence in the effect measures.

Fig. 1
Fig. 1 Flowchart according to Preferred Reporting Items for Systematic Reviews and Meta-Analysis (PRISMA) methodology et al. 2017 [50] USA Number of patients: not specified Number of learners: 4 Healthcare professionals: Neurology residents (first years-PGY-1) Hospital: simulation-based learning environment Simulation-based learning scenarios were developed, using a manikin controlled by the simulation lab personnel.Participants were videotaped performing the scenario, which was incorporated into their debrief One-off simulation training session Door-to-needle time 12.0 Ruff et al. 2017 [51] USA Number of patients: • Pre-simulation training: 72 patients • Post-simulation training: 98 patients Number of learners: 15 Healthcare professionals: Neurology Residents (PGY-2, PGY-3, and PGY-4) Hospital: emergency department Case-based simulation course with Socratic presentation of acute stroke cases (based on 4 scenarios).The boot camp was facilitated by senior residents, stroke fellows, and stroke attending physicians One-off resident bootcamp Door-to-needle time a Improvement in intravenous thromtechnicians) participated in on-site stroke team simulation training from seven hospitals.152 healthcare professionals completed the questionnaires from 6 stroke units (University Hospital Frankfurt did not participate) Hospitals: (Interdisciplinary Neurovascular Network)-three comprehensive and four regional stroke units (total of seven stroke units) Train-the-trainer seminar to educate stroke team trainers for each stroke unit conveying the principles of Crew Resource Management Simulation-based training with the simulation team in each participating hospital using a high-fidelity manikin.The remote-controlled highfidelity manikin, connected to a lifelike monitor, was filled with artificial blood, and placed on a stretcher, mimicking stroke-like symptoms Two and a half hours of stroke team training over 2 months throughout the seven participating hospitals Door-to-needle time a Improvement in intravenous thrombolysis knowledge Self-perceived usefulness of simulation training Feeling safe making decisions in acute stroke care 13.0 Tse-Chang et al. 2017 [53] USA Number of patients: not specified Number of learners: 187 Healthcare professionals: 153 emergency department nurses, 8 emergency department physicians, 6 neurologists, 4 pharmacists, 6 radiology technicians, and 10 phlebotomists Hospital: Emergency Department The simulation training included code stroke responders (five nurses) and the scenarios demonstrated right hemispheric syndrome.Participants in the simulation took a focused history and relayed their findings to the neurologist to evaluate the inclusion/exclusion criteria and the administration of thrombolysis One-off simulation training session with a 90-min scenario Door-to-needle time 12.0 Windle et al. 2017 [54] Australia Number of patients: not specified Number of learners: not specified Healthcare professionals: Stroke team, emergency, and clinical education clinicians, in addition to radiology, anaesthetics, administration and communications staff Hospital: tertiary centre.Off-site (hospital simulation centre) and in-situ simulation (emergency department) Phase I of the simulation training included patient scenario for the stroke team Phase II was the in-situ simulation in the emergency department; video footage was used to guide improvement Four off-site simulations and one insitu simulation for stroke education Door-to-needle time 10.5

Fig. 5 Fig. 6
Fig. 5 Random-effects meta-analysis of improvement in acute stroke knowledge pre-and post-simulation training

Fig. 7 Fig. 8
Fig. 7 Publication bias of eleven studies included in the meta-analysis of door-to-needle time

Table 2
Characteristics of included studies

Table 2 (
continued) a Door-to-needle times were obtained from hospital stroke/thrombolysis registry

Table 3
Secondary outcome measures post-simulation training (A) perceived usefulness of training (B) improvement in communication